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Abstract 


This document specifies the requirements and reference framework for Ethernet VPN (EVPN) 
Operations, Administration, and Maintenance (OAM). The requirements cover the OAM aspects 
of EVPN and Provider Backbone Bridge EVPN (PBB-EVPN). The framework defines the layered 
OAM model encompassing the EVPN service layer, network layer, underlying Packet Switched 
Network (PSN) transport layer, and link layer but focuses on the service and network layers. 
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1. Introduction 


This document specifies the requirements and defines a reference framework for Ethernet VPN 
(EVPN) Operations, Administration, and Maintenance (OAM) [RFC6291]. In this context, we use 
the term "EVPN OAM" to loosely refer to the OAM functions required for and/or applicable to 
[RFC7432] and [RFC7623]. 


EVPN is a Layer 2 VPN (L2VPN) solution for multipoint Ethernet services with advanced 
multihoming capabilities that uses BGP for distributing Customer/Client Media Access Control (C- 
MAC) address reachability information over the core MPLS/IP network. 


PBB-EVPN combines Provider Backbone Bridging (PBB) [IEEE-802.1Q] with EVPN in order to 
reduce the number of BGP MAC advertisement routes; provide client MAC address mobility using 
C-MAC [RFC7623] aggregation and Backbone MAC (B-MAC) [RFC7623] sub-netting; confine the 
scope of C-MAC learning to only active flows; offer per-site policies; and avoid C-MAC address 
flushing on topology changes. 


This document focuses on the fault management and performance management aspects of EVPN 
OAM. It defines the layered OAM model encompassing the EVPN service layer, network layer, 
underlying Packet Switched Network (PSN) transport layer, and link layer but focuses on the 
service and network layers. 


1.1. Relationship to Other OAM Work 


This document leverages concepts and draws upon elements defined and/or used in the 
following documents: 


[RFC6136] specifies the requirements and a reference model for OAM as it relates to L2VPN 
services, pseudowires, and associated Packet Switched Network (PSN) tunnels. This document 
focuses on Virtual Private LAN Service (VPLS) and Virtual Private Wire Service (VPWS) solutions 
and services. 


[RFC8029] defines mechanisms for detecting data plane failures in MPLS Label Switched Paths 
(LSPs), including procedures to check the correct operation of the data plane as well as 
mechanisms to verify the data plane against the control plane. 
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[IEEE-802.1Q] specifies the Ethernet Connectivity Fault Management (CFM) protocol, which 
defines the concepts of Maintenance Domains, Maintenance Associations, Maintenance End 
Points, and Maintenance Intermediate Points. 


[Y.1731] extends Connectivity Fault Management in the following areas: it defines fault 
notification and alarm suppression functions for Ethernet and specifies mechanisms for Ethernet 
performance management, including loss, delay, jitter, and throughput measurement. 


1.2. Specification of Requirements 


The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD 
NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to 
be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in 
all capitals, as shown here. 


1.3. Terminology 


This document uses the following terminology, much of which is defined in [RFC6136]: 


CE Customer Edge device; for example, a host, router, or switch. 
CFM Connectivity Fault Management [IEEE-802.1Q] 
DF Designated Forwarder [RFC7432] 


Down MEP A MEP that originates traffic away from and terminates traffic towards the core of 
the device in whose port it is logically located. 


EVI An EVPN instance spanning the Provider Edge (PE) devices participating in that 
EVPN [RFC7432]. 


L2VPN Layer 2 VPN 
LOC Loss of continuity 


MA Maintenance Association; a set of MEPs belonging to the same Maintenance Domain 
(MD) established to verify the integrity of a single service instance [[EEE-802.1Q]. 


MD Maintenance Domain; an OAM Domain that represents a region over which OAM 
frames can operate unobstructed [[EEE-802.1Q]. 


MEP Maintenance End Point; it is responsible for origination and termination of OAM 
frames for a given MA. A MEP is logically located in a device's port [IEEE-802.1Q]. 


MIP Maintenance Intermediate Point; it is located between peer MEPs and can process 
and respond to certain OAM frames but does not initiate them. A MIP is logically 
located in a device's port [[EEE-802.1Q]. 


MP2P Multipoint to Point 


NMS Network Management Station [RFC6632] 
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P Provider network interior (non-edge) node 
P2MP Point to Multipoint 

PBB Provider Backbone Bridge [RFC7623] 

PE Provider Edge network device 


Up MEP A MEP that originates traffic towards and terminates traffic from the core of the 
device in whose port it is logically located. 


VPN Virtual Private Network 


2. EVPN OAM Framework 


2.1. OAM Layering 


Multiple layers come into play for implementing an L2VPN service using the EVPN family of 
solutions as listed below. The focus of this document is the service and network layers. 


e The service layer runs end to end between the sites or Ethernet segments that are being 
interconnected by the EVPN solution. 

e The network layer extends between the EVPN PE (Provider Edge) nodes and is mostly 
transparent to the P (provider network interior) nodes (except where flow entropy comes 
into play). It leverages MPLS for service (i.e., EVI) multiplexing and split-horizon functions. 

e The transport layer is dictated by the networking technology of the PSN. It may be based on 
either MPLS LSPs or IP. 

e The link layer is dependent upon the physical technology used. Ethernet is a popular choice 
for this layer, but other alternatives are deployed (e.g., Packet over SONET (POS), Dense 
Wavelength Division Multiplexing (DWDM), etc.). 


This layering extends to the set of OAM protocols that are involved in the ongoing maintenance 
and diagnostics of EVPN networks. Figure 1 below depicts the OAM layering and shows which 
devices have visibility into what OAM layer(s). 
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+---+ +---+ 
+--+ l | +---+ +---+ +---+ | l +--+ 
CE eE E E a EE 
+--+ | | +---+ +---+ +---+ | | +--+ 
+---+ +---+ 
Oe Oa Service OAM ----------- OE (0) 
O Network OAM ----------- (0) 
Oo----o 0-------- 0--- 0------—- o Transport OAM 
O----O O----O 0O----O 0----0  O----O  0----o Link OAM 


Figure 1: OAM Layering 


Service OAM and Network OAM mechanisms only have visibility to the PE nodes but not the P 
nodes. As such, they can be used to deduce whether the fault is in the customer's own network, 
the local CE-PE segment, the PE-PE segment, or the remote CE-PE segment(s). EVPN Transport 
OAM mechanisms can be used for fault isolation between the PEs and P nodes. 


Figure 2 below shows an example network where Ethernet domains are interconnected via 
EVPN using MPLS, and it shows the OAM mechanisms that are applicable at each layer. The 
details of the layers are described in the sections below. 


+---+ +---+ 
+--+ l | +---+ +---+ +---+ | l +--+ 
E a 
+--+ | | +---+ +---+ +---+ | | +--+ 
+---+ +---+ 
Oa Or CFM (Service OAM) ---------- 0----0O 
O EVPN Network OAM --------- (0) 
Qmm- nnn (aan Oa as Q=- o MPLS OAM 
O----O O----O O----O O----O O----O o a 882.3 OAM 


[ IEEE-802 .3] 


Figure 2: EVPN OAM Example 


2.2. EVPN Service OAM 


The EVPN Service OAM protocol depends on what service-layer technology is being 
interconnected by the EVPN solution. In the case of [RFC7432] and [RFC7623], the service layer is 
Ethernet; hence, the corresponding Service OAM protocol is Ethernet CFM [IEEE-802.1Q]. 


EVPN Service OAM is visible to the CEs and EVPN PEs but not to the P nodes. This is because the 
PEs operate at the Ethernet MAC layer in [RFC7432] and [RFC7623], whereas the P nodes do not. 
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The EVPN PE MUST support MIP functions in the applicable Service OAM protocol (for example, 
Ethernet CFM). The EVPN PE SHOULD support MEP functions in the applicable Service OAM 
protocol. This includes both Up and Down MEP functions. 


As shown in Figure 3, the MIP and MEP functions being referred to are logically located within 
the device's port operating at the customer level. (There could be MEPs/MIPs within PE ports 
facing the provider network, but they would not be relevant to EVPN Service OAM as the traffic 
passing through them will be encapsulated/tunneled, so any customer-level OAM messages will 
just be treated as data.) Down MEP functions are away from the core of the device while Up MEP 
functions are towards the core of the device (towards the PE forwarding mechanism in the case 
of a PE). OAM messages between the PE Up MEPs shown are a type of EVPN Network OAM, while 
such messages between the CEs or from a PE to its local CE or to the remote CE are Service OAMs. 


+------- + (Ss SSSSS SSS See + SSeS ae oo oS saa + Sooo + 
|+----- +| |+-------------- +| |+-------------- +| |+----- +| 
I| CE || || PE | | FES s al ice K 
|+--+--+| |+---+-------- +-+| | t-+-------- +---+| |+--+--+| 
| | | | | | | | | | | | | 
|+--+--+| |+---+----- + l | +----- +---+| |+--+--+| 
|| MEP || | | er | | | | Up *| || || MEP || 
| |DownV | | | |MIP|MEP | | | |MEP |MIP| | | |DownV | | 
|+--+--+| | | | DownV | | | | DownV | | | |+--+--+| 
| | | et a a | Pel] A n Fas Nil | | | 
+---|---+ +----|-------- |--+ +--|-------- |----+ +---|---+ 
| | | | | | 
+o + +--- ... ---+ poe + 


Figure 3: CFM Details 


The EVPN PE MUST, by default, learn the MAC address of locally attached CE MEPs by snooping 
on CFM frames and advertising them to remote PEs as a MAC/IP Advertisement route. Some 
means to limit the number of MAC addresses that a PE will learn SHOULD be implemented. 


The EVPN PE SHOULD advertise any MEP/MIP local to the PE as a MAC/IP Advertisement route. 
Since these are not subject to mobility, they SHOULD be advertised with the static (sticky) bit set 
(see Section 15.2 of [RFC7432]). 


2.3. EVPN Network OAM 


EVPN Network OAM is visible to the PE nodes only. This OAM layer is analogous to Virtual Circuit 
Connectivity Verification (VCCV) [RFC5085] in the case of VPLS/VPWS. It provides mechanisms to 
check the correct operation of the data plane as well as a mechanism to verify the data plane 
against the control plane. This includes the ability to perform fault detection and diagnostics on: 


e the MP2P tunnels used for the transport of unicast traffic between PEs. EVPN allows for three 
different models of unicast label assignment: label per EVI, label per <ESI, Ethernet Tag>, and 
label per MAC address. In all three models, the label is bound to an EVPN Unicast Forwarding 
Equivalence Class (FEC). EVPN Network OAM MUST provide mechanisms to check the 
operation of the data plane and verify that operation against the control plane view. 
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e the MP2P tunnels used for aliasing unicast traffic destined to a multihomed Ethernet 
segment. The three label assignment models, discussed above, apply here as well. In all three 
models, the label is bound to an EVPN Aliasing FEC. EVPN Network OAM MUST provide 
mechanisms to check the operation of the data plane and verify that operation against the 
control plane view. 


e the multicast tunnels (either MP2P or P2MP) used for the transport of broadcast, unknown 
unicast, and multicast traffic between PEs. In the case of ingress replication, a label is 
allocated per EVI or per <EVI, Ethernet Tag> and is bound to an EVPN Multicast FEC. In the 
case of Label Switched Multicast (LSM) and, more specifically, aggregate inclusive trees, 
again, a label may be allocated per EVI or per <EVI, Ethernet Tag> and is bound to the tunnel 
FEC, 


e the correct operation of the Ethernet Segment Identifier (ESD split-horizon filtering function. 
In EVPN, a label is allocated per multihomed Ethernet segment for the purpose of 
performing the access split-horizon enforcement. The label is bound to an EVPN Ethernet 
segment. 

e the correct operation of the Designated Forwarder (DF) [RFC7432] filtering function. EVPN 
Network OAM MUST provide mechanisms to check the operation of the data plane and verify 
that operation against the control plane view for the DF filtering function. 


EVPN Network OAM mechanisms MUST provide in-band monitoring capabilities. It is desirable, 
to the extent practical, for OAM test messages to share fate with data messages. Details of how to 
achieve this are beyond the scope of this document. 


EVPN Network OAM SHOULD provide both proactive and on-demand mechanisms of monitoring 
the data plane operation and data plane conformance to the state of the control plane. 


2.4. Transport OAM for EVPN 


The Transport OAM protocol depends on the nature of the underlying transport technology in 
the PSN. MPLS OAM mechanisms [RFC8029] [RFC6425] as well as ICMP [RFC0792] and ICMPv6 
[RFC4443] are applicable, depending on whether the PSN employs MPLS or IP transport, 
respectively. Furthermore, Bidirectional Forwarding Detection (BFD) mechanisms per [RFC5880], 
[RFC5881], [RFC5883], and [RFC5884] apply. Also, the BFD mechanisms pertaining to MPLS-TP 
LSPs per [RFC6428] are applicable. 


2.5. Link OAM 


Link OAM depends on the data-link technology being used between the PE and P nodes. For 
example, if Ethernet links are employed, then Ethernet Link OAM ([IEEE-802.3], Clause 57) may 
be used. 


2.6. OAM Interworking 


When interworking two networking domains, such as actual Ethernet and EVPN to provide an 
end-to-end emulated service, there is a need to identify the failure domain and location, even 
when a PE supports both the Service OAM mechanisms and the EVPN Network OAM 
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mechanisms. In addition, scalability constraints may not allow the running of proactive 
monitoring, such as Ethernet Continuity Check Messages (CCMs) [IEEE-802.1Q], at a PE to detect 
the failure of an EVI across the EVPN domain. Thus, the mapping of alarms generated upon 
failure detection in one domain (e.g., actual Ethernet or EVPN network domain) to the other 
domain is needed. There are also cases where a PE may not be able to process Service OAM 
messages received from a remote PE over the PSN even when such messages are defined, as in 
the Ethernet case, thereby necessitating support for fault notification message mapping between 
the EVPN Network domain and the Service domain. 


OAM interworking is not limited, though, to scenarios involving disparate network domains. It is 
possible to perform OAM interworking across different layers in the same network domain. In 
general, alarms generated within an OAM layer, as a result of proactive fault detection 
mechanisms, may be injected into its client-layer OAM mechanisms. This allows the client-layer 
OAM to trigger event-driven (i.e., asynchronous) fault notifications. For example, alarms 
generated by the Link OAM mechanisms may be injected into the Transport OAM layer, and 
alarms generated by the Transport OAM mechanism may be injected into the Network OAM 
mechanism, and so on. 


EVPN OAM MUST support interworking between the Network OAM and Service OAM 
mechanisms. EVPN OAM MAY support interworking among other OAM layers. 


3. EVPN OAM Requirements 


This section discusses the EVPN OAM requirements pertaining to fault management and 
performance management. 


3.1. Fault Management Requirements 


3.1.1. Proactive Fault Management Functions 


The network operator configures proactive fault management functions to run periodically. 
Certain actions (for example, protection switchover or alarm indication signaling) can be 
associated with specific events, such as entering or clearing fault states. 


3.1.1.1. Fault Detection (Continuity Check) 


Proactive fault detection is performed by periodically monitoring the reachability between 
service end points, i.e., MEPs in a given MA, through the exchange of CCMs [IEEE-802.1Q]. The 
reachability between any two arbitrary MEPs may be monitored for: 


e in-band, per-flow monitoring. This enables per-flow monitoring between MEPs. EVPN 
Network OAM MUST support fault detection with per-user flow granularity. EVPN Service 
OAM MAY support fault detection with per-user flow granularity. 

e a representative path. This enables a liveness check of the nodes hosting the MEPs, assuming 
that the loss of continuity (LOC) to the MEP is interpreted as a failure of the hosting node. 
This, however, does not conclusively indicate liveness of the path(s) taken by user data 
traffic. This enables node failure detection but not path failure detection through the use of a 
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test flow. EVPN Network OAM and Service OAM MUST support fault detection using test 
flows. 


e all paths. For MPLS/IP networks with ECMP, the monitoring of all unicast paths between 
MEPs (on non-adjacent nodes) may not be possible since the per-hop ECMP hashing behavior 
may yield situations where it is impossible for a MEP to pick flow entropy characteristics 
that result in exercising the exhaustive set of ECMP paths. The monitoring of all ECMP paths 
between MEPs (on non-adjacent nodes) is not a requirement for EVPN OAM. 


The fact that MPLS/IP networks do not enforce congruency between unicast and multicast paths 
means that the proactive fault detection mechanisms for EVPN networks MUST provide 
procedures to monitor the unicast paths independently of the multicast paths. This applies to 
EVPN Service OAM and Network OAM. 


3.1.1.2. Defect Indication 

Defect indications can be categorized into two types: forward and reverse, as described below. 
EVPN Service OAM MUST support at least one of these types of event-driven defect indications 
upon the detection of a connectivity defect. 


3.1.1.2.1. Forward Defect Indication (FDI) 

FDI is used to signal a failure that is detected by a lower-layer OAM mechanism. A server MEP 
(i.e., an actual or virtual MEP) transmits a forward defect indication in a direction away from the 
direction of the failure (refer to Figure 4 below). 


Failure 
| 
+----- + +----- + V +----- + +----- + 
E e e Ce | D | 
+----- + +----- + +----- + +----- + 
<===========] | ============> 
Forward Forward 
Defect Defect 
Indication Indication 


Figure 4: Forward Defect Indication 


Forward defect indication may be used for alarm suppression and/or for the purpose of 
interworking with other layer OAM protocols. Alarm suppression is useful when a transport- 
level or network-level fault translates to multiple service- or flow-level faults. In such a scenario, 
it is enough to alert a network management station (NMS) of the single transport-level or 
network-level fault in lieu of flooding that NMS with a multitude of Service or Flow granularity 
alarms. EVPN PEs SHOULD support forward defect indication in the Service OAM mechanisms. 


3.1.1.2.2. Reverse Defect Indication (RDI) 


RDI is used to signal that the advertising MEP has detected a LOC defect. RDI is transmitted in the 
direction of the failure (refer to Figure 5). 
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Failure 
| 
+----- + +----- + V +----- + +----- + 
ere PB eXXXar| 1 Oleccaz | D | 
+----- + +----- + +----- + +----- + 
| ===========> <==========E | 
Reverse Reverse 
Defect Defect 
Indication Indication 


Figure 5: Reverse Defect Indication 


RDI allows single-sided management, where the network operator can examine the state of a 
single MEP and deduce the overall health of a monitored service. EVPN PEs SHOULD support 
reverse defect indication in the Service OAM mechanisms. This includes both the ability to signal 
a LOC defect to a remote MEP as well as the ability to recognize RDI from a remote MEP. Note 
that, in a multipoint MA, RDI is not a useful indicator of unidirectional fault. This is because RDI 
carries no indication of the affected MEP(s) with which the sender had detected a LOC defect. 


3.1.2. On-Demand Fault Management Functions 


On-demand fault management functions are initiated manually by the network operator and 
continue for a bounded time period. These functions enable the operator to run diagnostics to 
investigate a defect condition. 


3.1.2.1. Connectivity Verification 


EVPN Network OAM MUST support on-demand connectivity verification mechanisms for unicast 
and multicast destinations. The connectivity verification mechanisms SHOULD provide a means 
for specifying and carrying the following in the messages: 


e variable-length payload/padding to test connectivity problems related to the Maximum 
Transmission Unit (MTU). 


e test frame formats as defined in Appendix C of [RFC2544] to detect potential packet 
corruption. 


EVPN Network OAM MUST support connectivity verification at per-flow granularity. This includes 
both user flows (to test a specific path between PEs) as well as test flows (to test a representative 
path between PEs). 


EVPN Service OAM MUST support connectivity verification on test flows and MAY support 
connectivity verification on user flows. 


For multicast connectivity verification, EVPN Network OAM MUST support reporting on: 


e the DF filtering status of a specific port(s) or all the ports in a given bridge domain. 
e the split-horizon filtering status of a specific port(s) or all the ports in a given bridge domain. 
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3.1.2.2. Fault Isolation 


EVPN OAM MUST support an on-demand fault localization function. This involves the capability 
to narrow down the locality of a fault to a particular port, link, or node. The characteristic of 
forward/reverse path asymmetry in MPLS/IP makes fault isolation a direction-sensitive 
operation. That is, given two PEs A and B, localization of continuity failures between them 
requires running fault-isolation procedures from PE A to PE B as well as from PE B to PE A. 


EVPN Service OAM mechanisms only have visibility to the PEs but not the MPLS or IP P nodes. As 
such, they can be used to deduce whether the fault is in the customer's own network, the local 
CE-PE segment, or a remote CE-PE segment(s). EVPN Network and Transport OAM mechanisms 
can be used for fault isolation between the PEs and P nodes. 


3.2. Performance Management 


Performance management functions can be performed both proactively and on demand. 
Proactive management involves a recurring function, where the performance management 
probes are run continuously without a trigger. We cover both proactive and on-demand 
functions in this section. 


3.2.1. Packet Loss 


EVPN Network OAM SHOULD provide mechanisms for measuring packet loss for a given service 
-- for example, [RFC7680] and [RFC6673]. 


Given that EVPN provides inherent support for multipoint-to-multipoint connectivity, packet loss 
cannot be accurately measured by means of counting user data packets. This is because user 
packets can be delivered to more PEs or more ports than are necessary (e.g., due to broadcast, 
unpruned multicast, or unknown unicast flooding). As such, a statistical means of approximating 
the packet loss rate is required. This can be achieved by sending "synthetic" OAM packets that are 
counted only by those ports (MEPs) that are required to receive them. This provides a statistical 
approximation of the number of data frames lost, even with multipoint-to-multipoint 
connectivity. 


3.2.2. Packet Delay and Jitter 


EVPN Service OAM SHOULD support measurement of one-way and two-way packet delay and 
delay variation (jitter) across the EVPN network. Measurement of one-way delay requires clock 
synchronization between the probe source and target devices. Mechanisms for clock 
synchronization are outside the scope of this document. Note that Service OAM performance 
management mechanisms defined in [Y.1731] can be used. See also [RFC7679], [RFC2681], and 
[RFC3393]. 


EVPN Network OAM MAY support measurement of one-way and two-way packet delay and delay 
variation (jitter) across the EVPN network. 
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4. Security Considerations 


EVPN OAM MUST prevent OAM packets from leaking outside of the EVPN network or outside 
their corresponding Maintenance Domain. This can be done for CFM, for example, by having 
MEPs implement a filtering function based on the Maintenance Level associated with received 
OAM packets. 


EVPN OAM SHOULD provide mechanisms for implementation and optional use to: 


e prevent denial-of-service attacks caused by exploitation of the OAM message channel (for 
example, by forging messages to exceed a Maintenance End Point's capacity to maintain 
state). 


e authenticate communicating end points (for example, MEPs and MIPs). 


5. IANA Considerations 


This document has no IANA actions. 
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